Integrate Osiris reader optimizations for streams (backport #14189) #14586
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This saves a system call by sending the frame header and the chunk header at the same time. References rabbitmq/osiris#192 (cherry picked from commit 885e89e)
Osiris can read ahead data in case of small chunks. This saves system calls and increases consumption rate dramatically for some streams. This is transparent for the stream protocol, but requires a small tweak for the stream queue type implementation (passing in the previous iterator when creating a new one). The read ahead is on by default but can be deactivated with to the new stream.read_ahead configuration entry (true / false). Co-authored-by: Karl Nilsson <[email protected]> References rabbitmq/osiris#192 (cherry picked from commit 9f162df)
(cherry picked from commit 025280e)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR integrates 2 optimizations from Osiris for stream consumption:
Requires to bump Osiris to 1.10.0.
References rabbitmq/osiris#192
This is an automatic backport of pull request #14189 done by Mergify.